The present invention relates to the compositing of graphical image data.
A real world object, such as a painting that is being viewed, usually exhibits color, the appearance of which depends in part on the frequency of incidental light reflected from the pigment of the painting. The real world object can also exhibit other characteristics such as, for example, transparency to a degree that depends in part on the amount of incidental light that passes through the object. The painting can be transparent if, for example, the medium of the painting is a sheet of acetate and the density of the pigment on the acetate is adequately sparse. One can consequently see objects underlying or behind the sheet of acetate. The transparency of a real world object is also referred to as its opacity.
In digital image processing, color and opacity can be represented by data. The data can be in any form suitable for computation and, furthermore, can have the property that colors and opacities can be interpolated either linearly or otherwise. Graphics or image processing computer programs, for example, Adobe® Photoshop®, available from Adobe Systems Incorporated, of San Jose, Calif., can represent color and opacity with a color value and an alpha value, respectively. An alpha value, or simply alpha, is a scalar value that typically varies between 1 and 0, inclusive. An alpha of 0 indicates total transparency. An alpha of 1 indicates complete opacity. When working with graphic elements described with a color value and an alpha value, i.e., a color-alpha pair, the color value can be undefined or arbitrary when the alpha value is 0.
In general, two graphic elements each having a color value and an alpha value can be composited to produce a third graphic element that has a resulting color value and a resulting alpha. Each of the graphic elements can be, but is not limited to being, an image layer, a region of an image layer, a vector object, or one or more pixels. An image layer may include one or more regions that may differ in color and opacity.
An image processing computer program can build a final image by compositing two or more graphic elements together. The graphic elements can be a stack or part of a stack of graphic elements. The graphic elements can be, for example, image layers, which can be thought of as stacked sheets of acetate. The density of the pigment on the acetate, as alluded to above, controls the opacity of the sheet and, thus, the extent to which that sheet obscures images of underlying sheets.
Each graphic element can include image data and compositing controls. The image data can be defined as an array of pixels, with each pixel having a color value and, optionally, an alpha. Similarly, if there is a mask, the mask can also be defined as array of pixels, with each pixel having an alpha. Alternatively, the image data or the mask or both can be defined analytically, for example, by using shape outlines, or by other functions which map positions to color values and alphas. In addition, the image data and any mask can be dynamic, that is, computed from other data at the time the graphic elements are composited. The compositing controls may include a transfer mode, also known as a blending mode. The transfer mode of an graphic element specifies how the color in the graphic element mixes with the color accumulated from the underlying graphic elements in the same image position. The transfer mode is a recipe for blending colors that can be expressed as a function taking a color from an upper graphic element, i.e., an upper color, and a color from a lower graphic element, i.e., a lower color, to produce a third color. Image processing computer programs can provide a wide variety of predefined transfer modes. Transfer modes should not be confused with the well-known Porter-Duff compositing operations, which define how alphas determine how much of the blended colors survive in the result, but which assume essentially normal mode color mixing. T. Porter and T. Duff, “Compositing Digital Images”, SIGGRAPH 84, pp. 253–59 (1984).
Table 1 lists examples of transfer mode functions. Each of the example transfer modes works componentwise. That is, each color component value is calculated independently of the other color components. However, in the general case, transfer mode functions need not be so limited.
TABLE 1T(L, U)FormulaNormal (L, U)= UMultiply (L, U)= L · UScreen (L, U)= U + L − U · L = 1 − (1 − U) · (1 − L)Difference (L, U)= |U − L|ColorDodge (L, U)= Min (1, L/(1 − U)), with 0/0 treated as 0ColorBurn (L, U)= 1 − Min (1, (1 − L)/U), with 0/0 treated as 0Darker (L, U)= Min (L, U)Lighter (L, U)= Max (L, U)Add (L, U)= Max (1, L + U)Overprint (L, U)= 1 − Add (1 − L, 1 − U)SoftLight (L, U)= L − (1 − 2 · U) · L · (1 − L) if U ≦ 0.5= L + (2 · U − 1) · (D (L) − L) if U ≧ 0.5,whereD (x)= ((16 · x − 12) · x + 4) · x if x ≦ 0.25= sqrt (x) if x ≧ 0.25
Transfer modes that have two parameters, for example, those listed above, can be extended by the addition of a scalar parameter referred to in this specification as pseudo-opacity. The extension results in a three-parameter transfer mode T(L, U, q) where q represents a transfer mode pseudo-opacity (“pseudo-opacity”). Pseudo-opacity determines the contribution that each of the input color-alpha pairs makes to the result of the compositing. Pseudo-opacity usually varies between 1 and 0, inclusive. The three-parameter transfer mode satisfies the following constraints:T(L,U,0)=LT(L,U,1)=T(L,U)
That is, if pseudo-opacity is zero, then the transfer mode returns the lower color. If pseudo-opacity is one, then the extended transfer mode acts like the corresponding two-parameter transfer mode.
It is also desirable to have the resulting colors be defined by a continuous function of q. One way to achieve these constraints is to define:T(L,U,q)=(1−q)L+qT(L,U)As can be seen from the above discussion, any transfer mode can be extended to support pseudo-opacity. The similarity of the above equation to the conventional alpha-blending equation is one reason why this additional parameter is called pseudo-opacity.
The above definition is not, however, the only way to extend a transfer mode to support pseudo-opacity. For example, one can define:Difference(L,U,q) as |L−qU|This definition satisfies the criteria established above but does not match the results given by the alpha-blending style equation shown above.
Some image processing computer programs, in addition to having predefined transfer modes, can also receive user input defining new transfer modes. One such system is described in commonly-owned U.S. patent application Ser. No. 09/008,269, filed on Jan. 16, 1998, incorporated here by this reference, which discloses methods and apparatus, including computer programs, for use by a user to create new transfer modes (called compound transfer modes) based on existing (or newly-created) transfer modes and a scalar parameter, as appropriate.
Graphic elements such as, for example, image layers, are generally composited in order from bottom to top. The general process for compositing an graphic element can begin with calculation of any dynamic data in the graphic element, such as the color values of the pixels in the image and the alphas of the pixels in the mask. Then, the alpha is determined for each pixel from the mask, the global opacity value, if any, and the image data. Finally, the color value of each pixel of the graphic element is combined with the color value of the corresponding pixel in an accumulation buffer to generate a new composited color value. The combination is controlled by the alpha of the pixel and the transfer mode.